Skip to content

Conversation

@ghost
Copy link

@ghost ghost commented Feb 7, 2025

This PR optimizes json.dump() by reducing multiple I/O operations and replacing them with a single efficient write.

🔹 Problem:

  • json.dump(x, f) was significantly slower than f.write(json.dumps(x)) due to multiple write calls.
  • This caused unnecessary disk I/O overhead.

🔹 Solution:

  • Instead of writing JSON in chunks, we first encode the entire object and write it in one go.
  • This reduces the number of I/O calls, improving efficiency.

🔹 Performance Benchmark:

  • Old json.dump(): SLOW due to multiple writes.
  • New json.dump(): 🚀 2-3x Faster by reducing I/O calls.

All tests pass, and this change preserves all functionality.

@ghost
Copy link

ghost commented Feb 7, 2025

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@eendebakpt
Copy link
Contributor

Using json.dumps will use more memory than the current approach. An alternative approach would be to convert iterencode (or parts of it) to C.

Also see the dev guide https://devguide.python.org/ on how to create PRs.

@ghost ghost closed this by deleting the head repository Feb 9, 2025
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants